Twitter hashtag analysis: #birdoftheyear OR #boty
Ben Anderson (@dataknut)
Last run at: 2018-10-01 16:00:35
1 Purpose
To extract and visualise tweets and re-tweets of #birdoftheyear OR #boty in September/October 2018.
Borrowing extensively from https://github.com/mkearney/rtweet
Th anaysis used rtweet to ask the Twitter search API to extract ‘all’ tweets with the #birdoftheyear OR #boty hashtag in the ‘recent’ twitterVerse.
It is therefore possible that not quite all tweets have been extracted although it seems likely that we have captured most recent human tweeting which was the main intention. Future work should instead use the Twitter streaming API.
2 Load Data
Load the data (pre-collected using https://github.com/dataknut/hashTagR/blob/master/dataProcessing/getBoTY.R).
The data has:
- 9,078 tweets (including 599 quotes and 6,627 re-tweets)
- from 1,858 tweeters
- between 2018-09-18 14:43:25 and 2018-10-01 02:56:04 (UTC).
3 Analysis
3.1 Tweets and Tweeters over time
Figure 3.1: Number of tweets and tweeters
Figure 3.1 shows the number of tweets and tweeters in the data extract by day. The quotes, tweets and re-tweets have been separated.
If you are in New Zealand and you are wondering why there are no tweets today (2018-10-01) the answer is that twitter data (and these plots) are working in UTC and (y)our today hasn’t started yet in UTC - but don’t worry, all the tweets are here. It’s just our old friend the timezone… :-)
3.2 Who’s tweeting?
Next we’ll try by screen name.
Figure 3.2: N tweets per day by screen name
Figure 3.2 is a really bad visualisation of all tweeters tweeting over time. Each row of pixels is a tweeter (the names are probably illegible) and a green dot indicates a few tweets in the given day while a red dot indicates a lot of tweets. We’ve used plotly::ggplotly() so you can hover over the data points but it’s still pretty messy.
So let’s re-do that for the top 50 tweeters so we can see their tweetStreaks (tm)…
Top tweeters:
| screen_name | nTweets |
|---|---|
| birdoftheyear | 477 |
| Forest_and_Bird | 146 |
| coolbiRdpics | 135 |
| mifflangstone | 129 |
| NatForsdick | 128 |
| testeeves | 124 |
| vote4kaki | 119 |
| jackcraw57 | 104 |
| thebushline | 73 |
| newzealandbirds | 72 |
| kiwilullaby | 58 |
| sgalla32 | 53 |
| votegannet | 53 |
| hugobrown | 52 |
| votebittern | 51 |
And their tweetStreaks are shown in 3.3…
Figure 3.3: N tweets per day minutes by screen name (top 50, reverse alphabetical)
Any twitterBots…?
3.3 Which birds are mentioned the most (by hashtag)
This is very quick and dirty but… Figure 3.4 plots the number of tweets for each concatenated hashtag string (non-separated hashtags) after removing tweets which have variants of #birdOfTheYear and #boty as the only hashtags and selecting the top 50. Tweets mentioning birds without using a #<birdName> hashtag will not show up; birds #mentioned in tweets with a lot of varying other #hashtags will be under-represented etc etc. This is a really imperfect measure of just about anything so #YMMV.
Figure 3.4: Top 50 hashtag strings
Table 3.2 shows a slightly more intelligible table of the same data summarised across all days to date. Note that this is not a true count of the mentions of a particular #hashtag - we would need to seperate the hashtags and then count unique hashtags for that. Work in progress…
| hashtags | nTweets |
|---|---|
| takayay | 917 |
| gannet | 100 |
| Vote4Kakī|kakī | 33 |
| Pukeko|Kiwi|Australia | 28 |
| MenInBlack|BIB | 24 |
| takahey|votetakahē | 22 |
| birds|birdwatching | 21 |
| StanyAgaty|KłamaćJakMorawiecki|PiS|DośćKłamstw|Podpaski | 19 |
| kereru|kakī | 19 |
| TeamKakī|kakī|genetic|genomic | 18 |
| voterowi | 18 |
| DammitGannet | 14 |
| doadivebombbro|dammitgannet | 14 |
| TeamKaki | 13 |
| littlepenguin | 13 |
| kotāre|kingfisher | 12 |
| seabirds|albatross | 12 |
| TeamBlackPetrel | 11 |
| TeamKakī | 11 |
| dammitgannet|doadivebombbro | 10 |
| VoteRuru | 9 |
| DopestBirds | 8 |
| BrotherOfTheYear|TokyoInternationalFilmFestival2018|BrotherOfTheYearPH | 6 |
| kakī | 6 |
| votetomtit|itsthetits | 6 |
| StanyAgaty|KłamaćJakMorawiecki|PiS|DośćKłamstw | 5 |
| Kakī | 4 |
| KŌTARE | 4 |
| TeamKakī|dopest | 4 |
| votetakahē|makebirdsroundagain | 4 |
| BIB | 3 |
| DidYouKnow|Travel|adventure | 3 |
| Penguin | 3 |
| Vote4Kakī | 3 |
| evidence|conservation|kakī | 3 |
| makebirdsroundagain|manyshapesareround|livelaughtakahē|takayay|votetakahē | 3 |
| BrotherOfTheYear|BrotherOfTheYearPH | 2 |
| KotaPrezesa | 2 |
| TeamTawaki | 2 |
| fallcolours|woodpecker|SundayFunday | 2 |
| kakī|BraidedRiver | 2 |
| Kaczyński | 1 |
| animatethelivings|canlılarıcanlamdıralım|muslumtekin_ogr|petsofinstagram|pets|petstagram|petslovers|bird | 1 |
| teamrockhopper | 1 |
4 About
Analysis completed in 7.01 seconds ( 0.12 minutes) using knitr in RStudio with R version 3.5.1 (2018-07-02) running on x86_64-apple-darwin15.6.0.
A special mention must go to https://github.com/mkearney/rtweet (Kearney 2018) for the twitter API interaction functions.
Other R packages used:
- base R - for the basics (R Core Team 2016)
- data.table - for fast (big) data handling (Dowle et al. 2015)
- readr - for nice data loading (Wickham, Hester, and Francois 2016)
- ggplot2 - for slick graphs (Wickham 2009)
- plotly - fancy, zoomable slick graphs (Sievert et al. 2016)
- rtweet - twitter API search (Kearney 2018)
- knitr & bookdown - to create this document (Xie 2016b,Xie (2016a))
References
Dowle, M, A Srinivasan, T Short, S Lianoglou with contributions from R Saporta, and E Antonyan. 2015. Data.table: Extension of Data.frame. https://CRAN.R-project.org/package=data.table.
Kearney, Michael W. 2018. Rtweet: Collecting Twitter Data. https://cran.r-project.org/package=rtweet.
R Core Team. 2016. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. https://www.R-project.org/.
Sievert, Carson, Chris Parmer, Toby Hocking, Scott Chamberlain, Karthik Ram, Marianne Corvellec, and Pedro Despouy. 2016. Plotly: Create Interactive Web Graphics via ’Plotly.js’. https://CRAN.R-project.org/package=plotly.
Wickham, Hadley. 2009. Ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York. http://ggplot2.org.
Wickham, Hadley, Jim Hester, and Romain Francois. 2016. Readr: Read Tabular Data. https://CRAN.R-project.org/package=readr.
Xie, Yihui. 2016a. Bookdown: Authoring Books and Technical Documents with R Markdown. Boca Raton, Florida: Chapman; Hall/CRC. https://github.com/rstudio/bookdown.
———. 2016b. Knitr: A General-Purpose Package for Dynamic Report Generation in R. https://CRAN.R-project.org/package=knitr.